Activation of a virtual object

ABSTRACT

A VR device ( 110 ) comprising a controller ( 910 ) configured to: present a Virtual Reality space ( 120 ) comprising at least one virtual object ( 130 ) being associated with a gesture for executing an action associated with said virtual object ( 130 ); determine that the virtual object ( 130 ) is in a Line Of View (LOV); and providing a graphical marking ( 140 ) of the virtual object ( 130 ); wherein the graphical marking includes an indication of the associated gesture.

TECHNICAL FIELD

This application relates to a method, a computer-readable medium and adevice for providing improved activation of a virtual object, and inparticular to a method, a computer-readable medium and a device foractivation of a virtual object in an augmented reality, a virtualreality or a mixed reality for providing a touchless gesture-based userinterface.

BACKGROUND

Virtual Reality, or VR, has been known since the late 1970s and manysolutions have been proposed for how to provide user input includingwearing special gloves or keypads, especially for handling, such as whenselecting and activating a virtual object. Some examples of such systemsare proposing to use a motion sensor (such as an accelerometer and/or agyroscope) for determining the movements of a user's head and therebytracking what the user is watching in the virtual reality.

However, as the user's head is used for input and the virtual reality isoften well encompassing and comprising more content than is displayed atan instance in time, and as the virtual reality often comprises manyobjects which may all be associated with different functions andgestures, users are often confused and find it difficult to handle suchsystems and to select and execute correct or desired functions, at leastbefore substantial training has been performed.

There is thus a need for a VR device that facilitates user's perceptionenabling them to navigate complex Virtual reality landscapes.

SUMMARY

It is an object of the teachings of this application to overcome theproblems listed above by providing a VR device comprising a controllerconfigured to: present a Virtual Reality space comprising at least onevirtual object being associated with a gesture for executing an actionassociated with said virtual object; determine that the virtual objectis in a Line Of View; and providing a graphical marking of the virtualobject; wherein the graphical marking includes an indication of theassociated gesture.

Such a VR device is enabled to facilitate human perception by providingan indication of what gesture is associated with what icon so thatcommands can be given swiftly and accurately.

In one embodiment, the controller is further configured to provide asee-through view. The see-through view prompts and instructs the user toinput a gesture and also provides (real time) feedback on the gesturebeing made.

In one embodiment, the VR device comprises a mobile communicationsterminal, such as a smartphone. In one embodiment, the VR devicecomprises an internet tablet or a (laptop) computer. In one embodiment,the VR device comprises a game console. In one embodiment, the VR devicecomprises a media device such as a television set or media system. Inone embodiment the VR device comprises a pair of Augmented Reality orVirtual Reality glasses.

It is also an object of the teachings of this application to overcomethe problems listed above by providing a method for use in a VR devicecomprising a display, said method comprising presenting a VirtualReality space comprising at least one virtual object being associatedwith a gesture for executing an action associated with said virtualobject; determining that the virtual object is in a Line Of View; andproviding a graphical marking of the virtual object; wherein thegraphical marking includes an indication of the associated gesture.

It is a further object of the teachings of this application to overcomethe problems listed herein by providing a computer readable mediumcomprising instructions that when loaded into and executed by acontroller, such as a processor, in a VR device cause the execution of amethod according to herein.

The inventors of the present invention have realized, after inventiveand insightful reasoning that by providing a graphical indication of anassociated gesture as a marking of a selectable virtual object, possiblyalong with an animation (through a morphing) of the gesture and feedbackof the gesture (through a see-trough view), a user's perception isfacilitated greatly into being able to associate a virtual object, suchas an icon, with a gesture, understanding how the gesture is to be madeand receiving feedback of his gesture without risking to lose focus orconfusing one icon with another as regards their associated gestures.The user is also not forced to look at different sections or portions ofthe display to perceive the same amount of information. Furthermore, nowritten text is required which reduces the need for optical orprescription glasses (which is sometimes a problem when wearing VRglasses).

Additionally, the concept taught herein saves display space.

Moreover, the concept taught herein reduces the power consumption andrequirements of a VR device.

It should be explicitly noted that the simplicity and elegancy of theproposed solution is at the core of this invention and provides for auser interface that greatly facilitates human perception.

The concepts taught in this application are also applicable to augmentedrealities and mixed realities. For the purpose of this applicationVirtual Reality will thus be taken to also include augmented realitiesand mixed realities.

Other features and advantages of the disclosed embodiments will appearfrom the following detailed disclosure, from the attached dependentclaims as well as from the drawings. Generally, all terms used in theclaims are to be interpreted according to their ordinary meaning in thetechnical field, unless explicitly defined otherwise herein.

All references to “a/an/the [element, device, component, means, step,etc.]” are to be interpreted openly as referring to at least oneinstance of the element, device, component, means, step, etc., unlessexplicitly stated otherwise. The steps of any method disclosed herein donot have to be performed in the exact order disclosed, unless explicitlystated.

BRIEF DESCRIPTION OF DRAWINGS

The invention will be described in further detail under reference to theaccompanying drawings in which:

FIG. 1 shows an example embodiment of the operation of a VR devicearranged to operate according to the teachings herein;

FIG. 2 shows an example embodiment of the operation of a VR devicearranged to operate according to the teachings herein;

FIG. 3 shows an example embodiment of the operation of a VR devicearranged to operate according to the teachings herein;

FIGS. 4A and 4B show an example embodiment of the operation of a VRdevice arranged to operate according to the teachings herein;

FIGS. 5A, 5B and 5C show an example embodiment of the operation of a VRdevice arranged to operate according to the teachings herein;

FIGS. 6A, 6B, 6C and 6D show an example embodiment of the morphology ofa selected virtual object done by a VR device arranged to operateaccording to the teachings herein;

FIG. 6E shows an example embodiment of providing feedback of asuccessfully received gesture associated with a selected virtual objectdone by a VR device arranged to operate according to the teachingsherein;

FIGS. 7A, 7B, 7C, 7D and 7E show example embodiments of the presentationof a selected virtual object done by a VR device arranged to operateaccording to the teachings herein;

FIG. 8 shows an example embodiment of the operation of a VR devicearranged to operate according to the teachings herein;

FIG. 9A is a schematic view of a VR device according to the teachingsherein;

FIG. 9B is a schematic view of the components of a VR device accordingto the teachings herein;

FIG. 10 is a schematic view of a computer-readable memory according tothe teachings herein; and

FIG. 11 shows a flowchart illustrating a general method according to anembodiment of the teachings herein.

DETAILED DESCRIPTION

The disclosed embodiments will now be described more fully hereinafterwith reference to the accompanying drawings, in which certainembodiments of the invention are shown. This invention may, however, beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein; rather, these embodiments areprovided by way of example so that this disclosure will be thorough andcomplete, and will fully convey the scope of the invention to thoseskilled in the art. Like numbers refer to like elements throughout.

An improved manner of selecting and activating an object will bedisclosed below with reference to the accompanying figures. The examplewill be illustrated focusing on the handling of virtual objects in aVirtual Reality, but it should be clear that the processing is performedin part or fully in a VR device comprising a controller as disclosedherein with reference to FIGS. 9A, and 9B or caused to be performed byexecuting instructions stored on a computer-readable medium as disclosedwith reference to FIG. 10.

FIG. 1 show an example embodiment of the operation of a VR system 100comprising a VR device 110 arranged to operate according to theteachings herein, wherein the VR device 110 is configured to present avirtual reality (VR). As stated above there will not be made anydifference between a virtual reality, an augmented reality and a mixedreality for the context of this application and they will be referred toall as a virtual reality.

The virtual reality is presented as a VR space 120. The VR space 120lacks physical dimension (other than as a display on the display devicebeing used) and is therefore indicated with dashed lines. As would beappreciated and understood, the virtual reality in the display space islarger than as illustrated, actually it would be all encompassing, andthe portion being presented is only the portion matching the user'scurrent Field Of View (FOV).

The virtual reality comprises one or several virtual objects 130 and inthe example of FIG. 1, three such virtual objects 130A-C are displayed,but it should be noted that any number of objects (including zero) maybe displayed at any time. The objects 130 may be associated with afunction. The function may be to open or activate a file for example, orit may be to execute a specific action. One example that will be usedthroughout this description is a virtual object representing an iconassociated with a function of playing a media file. As a skilled personwould appreciate, the number of possible combinations and variations ofthe actual objects and possible associated functions is too large to beadequately disclosed in a patent application and any such attempt wouldpose an undue burden on the Applicant.

The user is of course looking in a specific or general direction in theVR space and this direction will hereafter be referred to as the user'sLine Of Sight or Line Of View (LOV). The Line Of View may be determinedby tracking the user's eye movements or as being a general directionfacing straight out into the VR space. The user can thus choose andselect a graphical object 130 by looking at or close to it. In oneembodiment the VR device is configured to only select and mark a virtualobject being looked at. In such an embodiment, the Line Of View may beconsidered to have a narrow angle of 0.5, 1, 1.5, 2, 3, 4 or up to 5degrees.

In one embodiment the VR device is configured to select and mark anyvirtual objects being in a vicinity or area that the user is looking at.In such an embodiment, the Line of View may be considered to have anarrow angle of 5, 10, 15, 20, 25 or up to 30 degrees.

The angle of the Line Of View may also be different in differentdimensions to correspond to the dimensions of the display space 120.Alternatively or additionally, such an area may be confined in space tofor example the middle 5%, 10%, 15% or 20% of the Field Of View. Thearea may be graphically marked to enable a user to direct it moreeasily, albeit at the price of the markings possibly cluttering theview. The VR device may be configured to only show the marking as avirtual object that is selectable is close to or within the area.

In the example of FIG. 2, the user has tilted his head, and consequentlythe VR device 110 and the Line Of View has shifted upwards and there arecurrently two virtual objects 130A and 130C being in the Line Of View ofthe user. The VR device is configured to mark or otherwise graphicallyhighlight those virtual objects 130A and 130C as being selectable forfurther activation. In the example of FIG. 2, this is done bygraphically changing (the edge of) the selectable virtual objects 130A,130C, which is illustrated by the dashed circles.

The VR device 110 according to the teachings herein is also configuredfor gesture control, and more precisely to detect and track an object infront of a camera and to determine if the tracked object performs aspecific gesture. Examples of such gestures are commonly known ingesture control systems and will not be explained in detailed herein.

However, it may be difficult for a user to remember all differentgestures and since gestures need to be easily discernible, they mightnot be logically (as in having a clear logical link) linked to theassociated function. Hence there is a problem that need to be solved byfacilitating the human perception of a user in order to enable the userto perform the correct gesture for activating the wanted selectedvirtual object.

The inventor of the present invention has realized that, in order tofacilitate the user's perception, the marking(s) of the virtualobject(s) selectable for further activation may be morphed into thecorresponding or associated gesture. The marking 140A, 140C, asillustrated in FIG. 3, will thus indicate the associated function andpossibly, if the morphing is animated, also how to perform the gesturecorrectly.

FIG. 3 shows an example of how the activation gestures of the virtualobjects 130A, 130C selectable for further activation are showngraphically as the markings 140A, 140C of the virtual objects 130A,130C.

The morphing is, in one embodiment, instantaneous whereby the graphicalindication of the gesture is shown immediately.

The morphing is, in one embodiment, animated whereby the graphicalindication of the gesture is shown as a morphing of the initial markinginto the finished gesture. The exact manner of morphing a marking into agesture depends on the marking and the gesture. However, as morphing isa common technique this will not be disclosed in detail and a skilledreader is directed to text books and other references for furtherdetails on such techniques.

FIGS. 4A and 4B show an example of a morphing from a circular marking toa hand showing an “OK” sign as a virtual object 130 is marked forpossible selection for further activation.

Returning to FIG. 3, a user's perception is thus facilitated regardingwhich virtual object(s) that may currently be selected for furtheractivation and which is the associated gesture that needs to beperformed, all while the user is simply looking in the general directionor at the virtual object(s). No other action is required at this stageby the user. The risk of the user changing focus while seeking to findfurther information is thus minimized. The user is also able to choosebetween more than one virtual object without further moving his head orother actions (apart from the actual selection gesture).

To further facilitate the user's perception both with regards to whichaction is to be taken and how this action is to be taken, the marking140 of the virtual object(s) 130 that is(are) possible to select forfurther activation may be graphically shown to comprise a see-throughfield of view of the area behind the VR device 110. In one embodimentthis see-through field of view may be the area immediately behind thevirtual object (which will be in front of the user, or in the user'sLine Of View). Or it may be an area surrounding a currently trackedobject. In the latter example, the user does not need to move his handsin front of him to perform the gesture, something that may be highlyappreciated in tight or crowded surroundings.

FIG. 5A shows an example of a virtual object 130 being marked (throughmarking 140) as selectable for further activation by the user directinghis line of view towards the virtual object 130. FIG. 5B shows how themarking 140 is morphed into a gesture, here being indicated in astylized manner as a U. FIG. 5C shows how the user's perception isfurther facilitated by also providing a see-through view 155 graphicallyconnected or overlaying the marking, thus being part of the marking 140.The see-through view 155 shows the tracked object 155 performing theassociated gesture. The user's perception has thus been facilitated inunderstanding which object to select and how, and also to perform theassociated action and correct any inadequacies of the performed gesturein real-time (as the gesture is clearly visible) without making multiplemovements or selections apart from gazing at or towards the virtualobject and performing the gesture.

FIGS. 6A to 6D shows how a virtual object 130 is marked and how themarking 140 is morphed into an associated gesture. From this, a skilledperson having read the contents of this application would realize how toimplement the morphing of the markings.

The VR device 110 may also be arranged to provide feedback that agesture associated with a virtual object has been successfully received,that is identified and matched to a virtual object. Apart from actuallyexecuting the associated action, the VR device 110 may thus provide agraphic (and/or audible) feedback to the user, but changing the markingor adding to it. FIG. 6E shows an example of how the marking 140 ischanged as a gesture has been identified. Some examples of how thefeedback may be given are, changing the edge or border of a graphicalrepresentation (135) of the virtual object 130, changing the marking140, adding a further marking (that may be seen as included in themarking 140, as marking 142 in FIG. 8), blinking, pulsating or otherwisehighlighting the marking 140 or the graphical representation 135 tomention a few examples.

FIGS. 7A to 7E shows alternatives of how to present the marking 140including the gesture indication and the see-through view of a virtualobject 130.

In FIG. 7A the marking 140 is as in FIG. 6 and also includes asee-through view 155 showing a tracked object (as a hand) 150, where thesee-through view is shown as partially or completely overlapping thevirtual object. A partial overlapping enables a user to associate thegesture with the graphical representation of the virtual object. Acomplete overlap saves display space. A compromise between the two maybe made by a system designer.

In FIG. 7B the see-through view 155 is displayed offset to a side orcorner of the virtual object.

In FIG. 7C the see-through view 155 is displayed transparently over thevirtual object 130. This enables for a higher degree of facilitating thehuman perception, but requires more computing power as it may bedifficult to properly isolate the tracked object from the background inorder to not clutter the display. Alternatively, the graphicalpresentation of the virtual object 130 may be transparently overlayed onthe see-through view, proving a high degree of association while notrequiring as much computing resources as isolating the tracked object.

In FIG. 7D both the graphical representation of the virtual object 135and the indication of the associated gesture 140 are displayed offset toa side or corner of the virtual object. In this embodiment, and as isshown in FIG. 7D, the see through view 155 and the tracked object 150may be displayed as a marked graphical representation of the virtualobject 130 as part of a general marking 145/155. As can be seen in FIG.7D, the graphical representation 135 and the indication of the gestureare shown in a smaller size. The size of the graphical markings,indications and see-through view may thus vary between embodiments.

In FIG. 7E both the see-through view 155 and the indication of theassociated gesture 140 are displayed as being the graphicalrepresentation 135 of the virtual object.

FIG. 8 shows a more detailed example of an embodiment as in FIG. 7Ewhere three virtual objects 130A-C are displayed. As a user's gaze isdirected towards or at a virtual object 130C, the object 130C is marked.In such an embodiment, the marking is provided by a see-through viewbeing overlayed with a graphical representation of the associatedgesture, the representation possibly being animated.

As is shown in FIG. 8, the remaining (visible) virtual objects 130A,130B may be marked as not being selectable, by changing for exampletheir colour, contrast, brightness and/or colour scheme (grey scale).

As mentioned in reference to FIG. 6E, a further graphical marking 142may be provided indicating the progress/success of the user providingand the VR device 110 receiving the gesture. In this example, thefurther marking 142 is a circle that is empty when no gesture isreceived and full when a gesture has been correctly received.

FIG. 9A generally shows a Virtual Reality (VR) device 110 according toan embodiment herein. In one embodiment the VR device 110 is configuredfor presenting a virtual reality to a user and allowing the user tooperate or interact with the virtual reality. A VR device 110 typicallycomprises a main portion arranged to be carried on a user's head, oftenin the form of eye wear, popularly called VR glasses. The VR device 110may thus comprise a strap 160 for holding the VR device 110 on a user'shead. The VR device 110 may also comprise speakers 165 for providingaudio as part of the VR experience. More prominently, the VR device 110comprises a VR display 170. The VR display 170 may be a specific displayintegral to the VR device 110. In one embodiment, as in the example ofFIG. 9A, the VR display 170 may comprise a holder 172 and an attachableVR display device 171. In such an embodiment, the VR display device 171may be seen as being the virtual device 110, as it is in fact, thisdevice that does most of the computing related to presenting the virtualreality.

I FIG. 9A an example of such a VR display device 171 is shownexemplified as a smartphone being one example of a mobile communicationsterminal. The description herein will be focused on the smartphone beingthe VR display 170 and no difference will be made between the VR display170 and the VR display device 171. Other examples of VR display devicesare a personal computers, desktop or laptop, an internet tablet, apersonal digital assistant and a game console.

Referring to FIG. 9A the mobile communications terminal in the form of asmartphone 170 comprises a housing 175 in which a display 125 isarranged. In one embodiment the display 125 is a touch display. In otherembodiments the display 125 is a non-touch display. Furthermore, thesmartphone 170 comprises two (physical) keys 133. In this embodimentthere are two keys 133, but any number of keys is possible and dependson the design of the smartphone 170. Examples of such (physical) keysare a mode key and an on/off key. In one embodiment the smartphone 170is configured to display a virtual object 130 on the display 125. Thevirtual object 130 may be associated with a functionality and thenconstitutes an icon or a virtual key. It should be noted that the numberof virtual objects, such as icons and virtual keys 130 are dependent onthe design of the smartphone 170 and an application that is executed onthe smartphone 170. The smartphone 170 is also equipped with a camera180.

The camera 180 is a digital camera that is arranged to take video orstill photographs by recording images on an electronic image sensor (notshown). In one embodiment the camera 180 is an external camera. In oneembodiment the camera is alternatively replaced by a source providing animage stream. The camera 180 is, in this embodiment, arranged to faceaway from the display 125, possibly being arranged on the backside ofthe smartphone, as indicated by the dashed lines.

The VR device 110 also comprises a motion sensor 185 configured todetect motion and movement of the VR device and hence the user to enablethe VR device to change what is being displayed accordingly. Such amotion sensor may be implemented through an accelerometer or agyroscope.

In one embodiment the VR device may comprise a further camera or othermeans for tracking eye movements, where the VR device is configured totrack the movement of the user's eyes and based on those eye movementsdetermine which icon or area that the user is looking at. Such detectionand tracking of eye movements can be used as an alternative to trackingthe movements of a user's head. Such detection and tracking of eyemovements can be used as a complement to support the tracking of themovements of a user's head. Such detection and tracking of eye movementscan be used as in addition to tracking the movements of a user's head.

In an embodiment of the VR device of FIG. 9A where the display is aspecific display and the VR device 110 does not comprise a smartphone171 or other such device to be attached or connected, the VR device 110also comprises a camera 180. In such an embodiment, the VR device 110also comprise essential components such as the controller 910, thememory 940 and the motion sensor 185.

The VR device 110 may further comprise an interface for connecting withother devices. Such interfaces may be wired or wireless. Some examplesof interfaces are USB (Universal Serial Bus) ports, Bluetooth™ ports,Ethernet ports or WiFi (according to IEEE standard 802.11) ports.

FIG. 9B shows a schematic view of the general structure of a VR deviceaccording to FIG. 9A. The VR device 110 comprises a controller 910 whichis responsible for the overall operation of the VR device 910 and ispreferably implemented by any commercially available CPU (“CentralProcessing Unit”), DSP (“Digital Signal Processor”) or any otherelectronic programmable logic device. The controller 910 is configuredto read instructions from the memory 940 and execute these instructionsto control the operation of the VR device 110. The memory 940 may beimplemented using any commonly known technology for computer-readablememories such as ROM, RAM, SRAM, DRAM, CMOS, FLASH, DDR, SDRAM or someother memory technology. The memory 940 is used for various purposes bythe controller 910, one of them being for storing application data andprogram instructions 950 for various software modules in the VR device110. The software modules include a real-time operating system, driversfor a user interface 920, an application handler as well as variousapplications 950.

The VR device 110 comprises a user interface 920, which in the VR deviceof FIG. 9A is comprised of the display 125, the keys 130, 133 and thecamera 180.

The VR device 110 may further comprise a communications (COM) interface930, which is adapted to allow the VR device to communicate with otherdevices through a radio frequency band technology or a wired technology.Examples of such technologies are USB, Ethernet, IEEE 802.11, IEEE802.15, ZigBee, WirelessHART, WIFI, Bluetooth®, W-CDMA/HSPA, GSM, UTRANand LTE to name a few.

The VR device 110 is further equipped with or arranged to be connectedto a camera 180. The camera 180 is a digital camera that is arranged totake video or still photographs by recording images on an electronicimage sensor (not shown).

The camera 180 is operably connected to the controller 910 to providethe controller with a video stream 965, i.e. the series of imagescaptured, for further processing possibly for use in and/or according toone or several of the applications 950.

In one embodiment the camera 180 is an external camera or source of animage stream. The camera may be two dimensional o three dimensional. Thecamera may also be a stereo camera or a field of flight camera. As askilled person would understand, there are many alternatives availablefor use as the camera.

Through the camera 180 (or rather the video stream) and the controller910, the VR device 110 (possibly comprising a smartphone 170 as in FIGS.9A and 9B), is configured to detect and track an object, such as a hand,via a video stream possibly provided by the camera 180. How such anobject H is detected and tracked is disclosed in the Swedish patentapplication SE 1250910-5 and will not be discussed in further detail inthe present application. For further details on this, please see thementioned Swedish patent application. It should be noted, however, thatthe teachings of the present application may be implemented through theuse of other tracking manners than disclosed in Swedish patentapplication SE 1250910-5.

FIG. 10 shows a schematic view of a computer-readable medium asdescribed in the above. The computer-readable medium 1000 is in thisembodiment a data disc 1000. In one embodiment the data disc 1000 is amagnetic data storage disc. The data disc 1000 is configured to carryinstructions 1001 that when loaded into a controller, such as aprocessor, executes a method or procedure according to the embodimentsdisclosed above. The data disc 1000 is arranged to be connected to orwithin and read by a reading device 1002, for loading the instructionsinto the controller. One such example of a reading device 1002 incombination with one (or several) data disc(s) 1000 is a hard drive. Itshould be noted that the computer-readable medium can also be othermediums such as compact discs, digital video discs, flash memories orother memory technologies commonly used.

The instructions 1001 may also be downloaded to a computer data readingdevice 1004, such as a smartphone or other device capable of readingcomputer coded data on a computer-readable medium, by comprising theinstructions 1001 in a computer-readable signal 1003 which istransmitted via a wireless (or wired) interface (for example via theInternet) to the computer data reading device 1004 for loading theinstructions 1001 into a controller. In such an embodiment thecomputer-readable signal 1003 is one type of a transientcomputer-readable medium 1000.

The instructions may be stored in a memory (not shown explicitly in FIG.10, but referenced 940 in FIG. 9B) of the smartphone 1004. Thesmartphone of FIG. 10 possibly being a smartphone as in FIGS. 9A and 9B.

References to computer program, instructions, code etc. should beunderstood to encompass software for a programmable processor orfirmware such as, for example, the programmable content of a hardwaredevice whether instructions for a processor, or configuration settingsfor a fixed-function device, gate array or programmable logic deviceetc.

References to ‘computer-readable storage medium’, ‘computer programproduct’, ‘tangibly embodied computer program’ etc. or a ‘controller’,‘computer’, ‘processor’ etc. should be understood to encompass not onlycomputers having different architectures such as single/multi-processorarchitectures and sequential (Von Neumann)/parallel architectures butalso specialized circuits such as field-programmable gate arrays (FPGA),application specific circuits (ASIC), signal processing devices andother devices. References to computer program, instructions, code etc.should be understood to encompass software for a programmable processoror firmware such as, for example, the programmable content of a hardwaredevice whether instructions for a processor, or configuration settingsfor a fixed-function device, gate array or programmable logic deviceetc.

FIG. 11 shows a flowchart of a general method according to the teachingsherein. A VR device displays 1110 one or more virtual objects anddetermines 1120 which virtual object(s) is/are currently in a Line OfView and marks 1130 these virtual objects. If an object is associatedwith a gesture for activating the virtual object, i.e. for causing anaction associated with the virtual object to be executed, the marking1130 includes providing 1131 an indication of the gesture, where theindication of the gesture is graphically linked to the marking (as inbeing part of the marking). The VR device may be configured to morph1132 the marking into the indication of the gesture. The VR device mayalso or alternatively be arranged to provide 1133 a see-through view aspart of the marking. For further details on these markings, reference ismade to FIGS. 1 to 8.

As a virtual object is determined to be selectable for activation, theVR device may be configured to activate 1140 the camera 180. By allowingthe camera to be in a sleep or inactive mode when no object associatedwith a gesture is selectable (i.e. not looked or gazed at) saves power.To further save power (and also bandwidth and memory space), the VRdevice may be configured to determine 1141 a type of the associatedgesture(s) and adapt the camera's setting based on the type of gesture.The gestures may be grouped into different types based on their dynamicsand/or complexity. For example a moving gesture (swiping hand) requiresmore frames (a higher frame rate) than a static gesture (“OK” sign)(lower frame rate). A higher frame rate may be set as 10 fps (frames persecond), 15 fps, 20 fps, 30 fps or higher. A lower frame rate may be setas 10 fps (frames per second), 7 fps, 5 fps or lower. A skilled readerwould understand that the exact frame rate is not the issue but that theframe rate associated with a first type is gesture is different than theframe rate associated with a second type of gestures. The VR device 110may thus adapt 1142 the camera's frame rate based on the gesture type.Similarly, the resolution used by the camera may also be set based onthe type of gesture. A gesture being graphically clean or uncomplicated(swiping hand) does not require a high resolution to be recognized(assuming the hand is easily discernible from the background), whereas agesture being graphically complicated (hand showing various numbers offingers) may require a high resolution to be recognized, especially ifthe background is cluttered. As for the frame rate, the exact resolutionused is not important for understanding the concept of this application,but that the resolution associated with a first type is gesture isdifferent than the resolution associated with a second type of gestures.The exact resolutions used also depend on the camera used. Some exampleof high resolutions are 4K, Ultra HD, HD, 720 p and some examples of lowresolutions are 720 p, 450 p, 330 p 250 p and 180 p. The VR device 110may thus adapt 1144 the camera's resolution based on the gesture type.

As the camera is activated, the VR device identifies and tracks anobject 1150, possibly by providing a see-through view of the trackedobject. As the object is tracked, the VR device is configured toidentify 1160 a gesture and to match this gesture to the associatedmarked virtual objects and in response thereto execute 1170 an actionassociated with the marked virtual object. Feedback regarding thesuccess or progress of the received gesture may also be provided 1161.

Of course, the VR device may also be configured to identify a gesture asbeing associated with an object not being marked or to a differentfunction or action, and in response thereto execute an associatedaction.

The teachings herein provide the benefit that a user's perception isfacilitated in that the user is enabled to associated a virtual objectwith an associated gesture and how the gesture is to be performed andalso with feedback on how the user is performing the gesture withouthaving to make unnecessary movements (such as moving the user's head)thereby reducing the risk of losing focus of a virtual object, confusingone object with another regarding their functionality, and forassociating the gesture with the virtual object for future reference.The user is also enabled to use the VR device, such as VR glasses,without glasses should the user wish so as no written information isneeded to be read.

The invention has mainly been described above with reference to a fewembodiments. However, as is readily appreciated by a person skilled inthe art, other embodiments than the ones disclosed above are equallypossible within the scope of the invention, as defined by the appendedpatent claims.

The invention claimed is:
 1. A VR device comprising a controllerconfigured to: present a Virtual Reality space, wherein said VirtualReality space relates to augmented reality or mixed reality andcomprises at least one virtual object being associated with a gesturefor executing an action associated with said virtual object; determinethat the virtual object is in a Line Of View; and providing a graphicalmarking of the virtual object, wherein the graphical marking includes anindication of the associated gesture; wherein the graphical marking ofthe virtual object is provided as a morphing into the indication of theassociated gesture, wherein the morphing is animated, whereby theindication of the associated gesture is shown as a morphing of thegraphical marking into the indication of the associated gesture, therebyreplacing the graphical marking with the indication of the associatedgesture.
 2. The VR device according to claim 1, wherein said controlleris further configured to provide determine that a virtual object is in aLine Of View of the virtual object falls within an angle of the Line OfView.
 3. The VR device according to claim 1, wherein said controller isfurther configured to provide a see-through view of a tracked object,said see-through view being included in the graphical marking of thevirtual object.
 4. The VR device according to claim 3, wherein saidcontroller is further configured to overlay the indication of thegesture with the see-through view.
 5. The VR device according topreceding claim 3, wherein said controller is further configured toprovide said see-though view as a graphical representation of thevirtual object in the Line Of View.
 6. The VR device according to claim1, wherein said controller is further configured to provide saidindication of said gesture at a side of the marking.
 7. The VR deviceaccording to claim 1, said VR device further comprising a camera,wherein said controller is further configured to activate the camera inresponse to determining that a virtual object is in the Line Of View. 8.The VR device according to claim 1, where the VR device furthercomprises a camera, wherein said controller is further configured todetermine a type of the associated gesture and adapt a resolution ofsaid camera based on the type of the associated gesture.
 9. The VRdevice according to claim 1, where the VR device further comprises acamera, wherein said controller is further configured to determine atype of the associated gesture, wherein said controller is furtherconfigured to adapt a frame rate of said camera based on the type of theassociated gesture.
 10. The VR device according to claim 1, wherein saidVR device is configured to be attached to a VR display device.
 11. TheVR device according to claim 10, wherein said VR display device is asmartphone.
 12. The VR device according to claim 1, wherein said VRdevice is a smartphone.
 13. The VR device according to claim 1, whereinsaid VR device comprises Virtual Reality glasses or Augmented Realityglasses.
 14. A method for use in a VR device, said method comprising:presenting a Virtual Reality space, wherein said Virtual Reality spacerelates to augmented reality or mixed reality and comprises at least onevirtual object being associated with a gesture for executing an actionassociated with said virtual object; determining that the virtual objectis in a Line Of View; and providing a graphical marking of the virtualobject, wherein the graphical marking includes an indication of theassociated gesture; wherein the graphical marking of the virtual objectis provided as a morphing into the indication of the associated gesture,wherein the morphing is animated, whereby the indication of theassociated gesture is shown as a morphing of the graphical marking intothe indication of the associated gesture, thereby replacing thegraphical marking with the indication of the associated gesture.
 15. Anon-transitory computer readable storage medium encoded withinstructions that, when loaded and executed in a controller, causes themethod according to claim 14 to be performed.